Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[bugfix](iceberg)fix datetime conversion error for 2.0 #35831

Merged
merged 1 commit into from
Jun 6, 2024
Merged

[bugfix](iceberg)fix datetime conversion error for 2.0 #35831

merged 1 commit into from
Jun 6, 2024

Conversation

wuwenchi
Copy link
Contributor

@wuwenchi wuwenchi commented Jun 4, 2024

Proposed changes

bp: #35708

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@wuwenchi
Copy link
Contributor Author

wuwenchi commented Jun 4, 2024

run buildall

@github-actions github-actions bot added the area/planner Issues or PRs related to the query planner label Jun 4, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 49519 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c984246a7c9d92126f96f23c37b83bf6d460d034, data reload: false

------ Round 1 ----------------------------------
q1	17800	4363	4295	4295
q2	2074	159	146	146
q3	10427	1905	1924	1905
q4	10357	1229	1302	1229
q5	8722	3863	3886	3863
q6	230	125	152	125
q7	2050	1590	1583	1583
q8	9281	2716	2689	2689
q9	10548	10375	10271	10271
q10	8604	3517	3514	3514
q11	426	242	259	242
q12	474	305	305	305
q13	18341	3978	3986	3978
q14	362	333	342	333
q15	519	462	456	456
q16	675	579	571	571
q17	1125	970	992	970
q18	7263	6936	6788	6788
q19	1730	1600	1472	1472
q20	519	336	300	300
q21	4485	4119	4093	4093
q22	509	406	391	391
Total cold run time: 116521 ms
Total hot run time: 49519 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4364	4278	4320	4278
q2	323	230	227	227
q3	4163	4152	4153	4152
q4	2764	2759	2764	2759
q5	7231	7111	7139	7111
q6	237	121	118	118
q7	3237	2788	2841	2788
q8	4377	4470	4495	4470
q9	16855	16772	16749	16749
q10	4253	4253	4307	4253
q11	761	657	716	657
q12	1042	855	859	855
q13	7042	3742	3768	3742
q14	474	420	436	420
q15	499	463	464	463
q16	745	690	677	677
q17	3813	3882	3852	3852
q18	8770	8816	8764	8764
q19	1715	1693	1652	1652
q20	2390	2116	2119	2116
q21	8401	8567	8411	8411
q22	1039	983	957	957
Total cold run time: 84495 ms
Total hot run time: 79471 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 203720 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c984246a7c9d92126f96f23c37b83bf6d460d034, data reload: false

query1	933	424	377	377
query2	6525	2737	2597	2597
query3	6916	212	210	210
query4	20082	18305	18123	18123
query5	19758	6511	6529	6511
query6	284	216	241	216
query7	4158	303	319	303
query8	268	244	252	244
query9	3187	2750	2654	2654
query10	422	295	297	295
query11	11384	10820	10843	10820
query12	124	81	71	71
query13	5590	695	678	678
query14	17675	13097	13820	13097
query15	365	218	247	218
query16	6459	298	265	265
query17	1687	1461	890	890
query18	2291	417	411	411
query19	210	153	149	149
query20	76	81	83	81
query21	191	96	97	96
query22	5280	4997	5030	4997
query23	32560	31930	31898	31898
query24	6973	6600	6660	6600
query25	538	426	425	425
query26	548	171	160	160
query27	1874	298	300	298
query28	6188	2363	2301	2301
query29	2904	2670	2721	2670
query30	248	168	166	166
query31	902	765	751	751
query32	68	62	63	62
query33	435	273	265	265
query34	869	481	496	481
query35	1135	925	917	917
query36	1340	1203	1232	1203
query37	94	64	59	59
query38	3134	2957	2899	2899
query39	1379	1324	1329	1324
query40	203	100	98	98
query41	41	38	38	38
query42	84	88	88	88
query43	683	691	737	691
query44	1142	710	728	710
query45	244	235	230	230
query46	1216	980	947	947
query47	1687	1828	1726	1726
query48	999	700	709	700
query49	620	378	371	371
query50	875	572	623	572
query51	4766	4655	4583	4583
query52	94	85	87	85
query53	443	329	317	317
query54	2666	2477	2487	2477
query55	92	77	86	77
query56	221	206	211	206
query57	1211	1191	1026	1026
query58	220	211	210	210
query59	4463	4121	4288	4121
query60	212	208	203	203
query61	99	99	111	99
query62	846	464	474	464
query63	493	338	346	338
query64	2616	1599	1513	1513
query65	3646	3539	3526	3526
query66	790	394	378	378
query67	16599	15448	15369	15369
query68	8979	664	643	643
query69	568	337	329	329
query70	1752	1383	1343	1343
query71	423	305	322	305
query72	6541	3502	3524	3502
query73	740	331	337	331
query74	6329	5894	5874	5874
query75	5369	3730	3792	3730
query76	5468	1106	1186	1106
query77	950	258	261	258
query78	12585	11481	11457	11457
query79	8605	651	630	630
query80	1614	413	395	395
query81	502	233	237	233
query82	1649	100	100	100
query83	191	135	134	134
query84	264	74	72	72
query85	869	323	325	323
query86	348	302	292	292
query87	3231	3021	3046	3021
query88	5085	2339	2330	2330
query89	400	295	296	295
query90	1946	208	218	208
query91	181	141	142	141
query92	63	52	52	52
query93	6319	595	576	576
query94	709	206	209	206
query95	1095	1068	1066	1066
query96	648	332	337	332
query97	6522	6397	6402	6397
query98	193	175	175	175
query99	2940	919	988	919
Total cold run time: 315403 ms
Total hot run time: 203720 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.22 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c984246a7c9d92126f96f23c37b83bf6d460d034, data reload: false

query1	0.02	0.02	0.02
query2	0.06	0.02	0.02
query3	0.26	0.05	0.05
query4	1.80	0.07	0.06
query5	0.54	0.52	0.52
query6	1.24	0.62	0.63
query7	0.02	0.01	0.01
query8	0.03	0.03	0.02
query9	0.54	0.51	0.47
query10	0.55	0.54	0.53
query11	0.12	0.09	0.09
query12	0.11	0.08	0.09
query13	0.62	0.62	0.61
query14	0.79	0.80	0.77
query15	0.80	0.76	0.76
query16	0.36	0.36	0.35
query17	1.00	1.01	1.01
query18	0.22	0.26	0.24
query19	1.93	1.83	1.88
query20	0.02	0.01	0.01
query21	15.74	0.56	0.55
query22	1.94	1.73	1.75
query23	17.19	0.95	0.87
query24	4.32	1.63	1.48
query25	0.38	0.06	0.07
query26	0.59	0.15	0.14
query27	0.05	0.04	0.03
query28	8.11	0.75	0.70
query29	12.62	2.38	2.30
query30	0.63	0.54	0.53
query31	2.81	0.39	0.37
query32	3.39	0.51	0.49
query33	3.05	3.07	3.09
query34	15.24	4.79	4.76
query35	4.87	4.85	4.83
query36	1.04	1.00	1.01
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.14
query41	0.07	0.01	0.01
query42	0.02	0.01	0.01
query43	0.03	0.01	0.02
Total cold run time: 103.4 s
Total hot run time: 31.22 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit c984246a7c9d92126f96f23c37b83bf6d460d034 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.7 seconds inserted 10000000 Rows, about 460K ops/s

@morningman morningman merged commit ceee062 into apache:branch-2.0 Jun 6, 2024
24 of 25 checks passed
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@wuwenchi
[bugfix](iceberg)fix datetime conversion error for 2.0 (apache#35831)
9bc870f 
bp:  apache#35708
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
area/planner Issues or PRs related to the query planner
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@wuwenchi @doris-robot @morningman